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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1 .17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.1 14. Applicant's submission filed on December 18, 2009 has been entered. 

Response to Arguments 

2. Applicant's arguments filed December 18, 2009 have been fully considered but they are 
not persuasive. 

3. With respect to Claims 9, 25, and 26, Applicant argues, "Applicants have amended the 
claims to make it clear that the present invention uses a database of image files with associated 
text and links, but in applicants invention the searching of files is done by searching the database 
using the image, where as Noda teaches taking an image, extracting characteristics of the image, 
use the extracted characteristics with user input of, for example, color of petals, shade of the 
color and type of flower, to search the database for like characteristics and returning an image 
that is associated with those like characteristic with a corresponding description. In other words, 
Noda is not searching the database using the image, but is searching the database by comparing 
the characteristics (See Figure 2)." 
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4. The Examiner respectfully disagrees with Applicant's position. It appears that Applicant 
is alleging Claim 9 now recites that the captured image must be compared to the existing images 
in the database by image content alone (i.e., without consideration of additional corresponding 
data attached to the image file). However, the Examiner respectfully notes Claim 9, both in lines 
4-7 and 14-17, does not specify such a specific feature. In fact, there is no recitation of how 
the images are determined to be found similar. 

5. The Examiner further notes Claim 9 interchangeably recites "image" and "image files." 
The Examiner respectfully notes there is a difference between comparing images and comparing 
image files. An example of an image file would be a JPEG image where the file consists of 
metadata and just plain image data, whereas an image is just plain image data with no 
corresponding data attached thereto. 

6. Applicant further argues, "Nowhere does Aarnio teach, nor is it obvious based on reading 
Aarnio, that one could return images for matching. It is our reading that Aarnio only discloses 
OCR matching as a means for image comparison. 

7. The Examiner respectfully disagrees with Applicant's position. OCR is a form analyzing 
the contents of an image. Therefore, Aarnio is performing image content based matching. 
Moreover, Aarnio teaches a method and apparatus for providing precise location information 
through a communications network (Figures 1 and 3). Aarnio further teaches that the database to 
be accessed is a computer network 1 8, which may be the Internet, or World Wide Web (column 
2, lines 57-60), which includes several databases that can then be checked. 

8. Finally, based on the structure and content of Applicant's arguments, it appears that 
Applicant is arguing in favor of features which are not claimed as specifically as the arguments 
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suggest. Applicant is reminded, although the claims are interpreted in light of the specification, 
limitations from the specification are not read into the claims. See In re Van Geuns, 988 
F.2d 1181, 26USPQ2d 1057 (Fed. Cir. 1993). 

9. For these reasons, the Examiner will maintain the prior art rejection. 

Claim Rejections - 35 USC § 112 

10. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

1 1 . Claims 9-24 are rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. 

12. With respect to independent Claim 9, line 13 recites "the database" when lines 5 and 12 
each recite "a database." It is not known whether the databases in lines 5 and 12 are the same 
databases or not. Therefore, simply referring to the "the database" Leaves Claim 9 indefinite. 

13. With respect to dependent Claims 10-24, by virtue of their dependency upon Claim 9 
are also indefinite. 

Claim Rejections - 35 USC §103 

14. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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15. Claims 9-22 and 24 - 28 are rejected under 35 U.S. C. 103(a) as being unpatentable 
over Noda et al. in view of Aarnio (US 6,522,889). 

NOTES: The Examiner notes Claims 9, 25, and 26 appear to be related as apparatus, 
storage medium, and method respectively. Accordingly, these claims will grouped together 
under one rejection using the language of Claim 9. Furthermore, the Examiner's above-response 
is fully incorporated in the rejections below. Finally, the Examiner notes that the storage 
medium claimed in Claim 25 is not described in the specification. Claim 25 has been directed 
towards a storage medium since its original filling. To be clear as to whether the "storage 
medium" is directed towards statutory subject matter, a paragraph should be added to 
specification including the types of statutory storage mediums that can be included. The 
paragraph should be consistent with the remainder of the specification and should not include 
any new matter. 

16. For Claims 9, 25, and 26, Noda et al. explicitly teach: 

A system to provide location awareness services comprising: 

(a) a handheld device (digital camera, PDA, PHS, Figure 1) comprising: 

(i) a camera to capture an image of an location (digital camera, Figure 1); 

(ii) a mobile communication device (PHS, Figure 1), coupled to the camera and to 
a wireless network (WWW sever, Figure 1), to communicate the image to a server 
having a database with existing image files to find similar images (page 25, 
abstract, lines 8-12) by locating and comparing the captured image with other 
existing images located in the database of existing image files; and 
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(iii) a user interface (PDA, Figure 1), coupled to the mobile communication 
device, to communicate to an user any resulting relevant images and associated 
hyperlinks of found similar images (page 28, 3.5 and Figure 5); and 

(b) a computer network including the wireless network (WWW server, Figure 1) and a 
wired network (flower database, Figure 2); 

(c) a server (see Figure 1), connected to the computer network having a database to store 
images of interest; the database having the plurality of files, each file including an image, text 
and an associated link; and 

(c) a plurality of computers (WWW server and flower database, Figure 1), each computer 
have a plurality of computer files in a format to display as a web page, each file including an 
image and textual information and connected to the computer network (process information files 
and flower information files respectively), at least one of the computer files having an associated 
hyperlink and including an image similar to the captured image and text describing an object in 
the image which when found by the server is added to the database (page 26, paragraph 3, lines 
8-13, see also Figure 5). 

The Examiner respectfully notes, as described by item 3 (Image Retrieval System of 
Flowers), Noda uses characteristics entered by a user to generate a binary image of the captured 
image (see 3.1), then calculates the characteristics of shape using a circularity measure (as 
defined by equation 1; see 3.2), then calculates the characteristics of color using a center of 
gravity calculation (see 3.3). Noda combines these characteristics into a table (see Table 1) and 
compares them to characteristics of other images in the database to find a match(s) (see 3.4). 
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Once a match(s) is found, the image(s) is displayed on a webpage and the matched image(s) and 
URL of the webpage is sent to the user (see 3.5 and 2). 

However, Noda et al. do not explicitly teach to search for additional images of interest 
located on other computers connected to the network when an image of interest is not located on 
the server. 

On the other hand, Aarnio teaches a method and apparatus for providing precise location 
information through a communications network (Figures 1 and 3). Aarnio further teaches that 
the database to be accessed is a computer network 18, which may be the Internet, or World Wide 
Web (column 2, lines 57-60), which includes several databases that can then be checked. 

Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include the computer network found in Aarnio as the database in Noda et 
al. in order to pinpoint an exact location of the image by using the vast sources of the internet 
(column 1, lines 35-39). 

17. As for Claim 10, Noda et al explicitly teach: 

The system to provide location awareness services as recited in Claim 9 wherein the user 
interface comprises a display (PDA, browser Figure 1, see also Figure 5). 

18. As for Claim 11, Noda et al explicitly teach: 

The system to provide location awareness services as recited in Claim 9 wherein the user 
interface comprises a computer connection (PDA connects to computers wirelessly through PHS, 
Figure 1). 

19. As for Claim 12, Noda et al. explicitly teach: 
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The system to provide location awareness services as recited in Claim 9 wherein the 
associated text describes features of the object in the image including geographical location of 
the object (page 28, 3.5). 

20. As for Claim 13, Noda et al. explicitly teach: 

The system to provide location awareness services as recited in Claim 9 wherein the 
associated text describes features of the object in the image including a description and historical 
facts regarding the object (page 28, 3.5). 

21. As for Claim 14, Noda et al. explicitly teach: 

The system to provide location awareness services as recited in Claim 9 wherein the 
associated text includes a uniform resource locator (URL) (Figure 5, the word detail is 
considered the URL because when it is clicked the web address corresponding to the link is 
opened and seen in Figure 6). 

22. As for Claim 15, Noda et al explicitly teach: 

The system to provide location awareness services as recited in Claim 9, wherein at least 
one of the computers includes a plurality of computer files having images of fields of interest 
(flower database Figure 1). 

However, Noda et al. do not explicitly teach that the database has information of 
locations of interest located within a predetermined radius about a geographical location. 

In the came field of endeavor, Aarnio teaches a method and apparatus for providing 
precise location information through a communications network (Figures 1 and 3). Aarnio 
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further teaches a server service 24 that provides information to the user about the surroundings in 
the geographical region (column 4, lines 33-38). 

Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include the information about the geographical surroundings found in 
Aarnio into the images found in the database of Noda et al. in order to provide a convenience for 
the user such as in a case described in column 4, lines 38-43. 

23. As for Claim 16, the combination of Noda et al. in view of Aarnio, as applied to claim 
15, further teaches the system to provide location awareness services as recited in Claim 15 
wherein the computer having a plurality of computer files having images of objects of interest 
located within a predetermined radius about a geographical location (column 4, lines 33-38, 
Aarnio) was previously trained to find common objects known to be of interest (page 26-28, 3- 
3.5 explains the finding of common images). 

24. As for Claim 17, Noda et al. explicitly teach: 

The system to provide location awareness services as recited in Claim 9 wherein at least 
one of the computers includes at least one computer file having an image of an object of known 
interest and an associated image of an object of less recognized interest within a predetermined 
radius about a geographical location of the known interest object (section 3.1 describes the 
extraction of the flower, which is the isolation of the image and at the same time the isolation of 
the surroundings, therefore it is inherent that the surroundings can be used to identify an object 
of less interest using the same methods describe in the experimentation with real images 4.1) to 
aid a user in finding the object of less recognized interest. 

25. As for Claim 18, Noda et al. explicitly teach: 
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The system to provide location awareness services as recited in Claim 9 wherein at least 
one of the computers includes at least one computer file having an image of an object of known 
interest and an associated image of an object of less recognized interest within the field of view 
of the known interest object (section 3.1 describes the extraction of the flower, which is the 
isolation of the image and at the same time the isolation of the surroundings, therefore it is 
inherent that the surroundings can be used to identify an object of less interest using the same 
methods describe in the experimentation with real images 4.1) to aid a user in finding the object 
of less recognized interest. 

26. As for Claim 19, Noda et al explicitly teach the system to provide location awareness 
services as recited in Claim 9, which includes at least one computer (flower database Figure 1). 

However, Noda et al. do not teach that the computer has the capability of searching other 
computers to find matching files. 

In the same field of endeavor, Aarnio teaches a method and apparatus for providing 
precise location information through a communications network (Figures 1 and 3). Aarnio 
further teaches that the database to be accessed is a computer network 1 8, which may be the 
Internet, or World Wide Web (column 2, lines 57-60), which includes several computers that can 
then be checked. 

Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include the computer network found in Aarnio as the database in Noda et 
al. in order to pinpoint an exact location of the image by using the vast sources of the internet 
(column 1, lines 35-39). 
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27. As for Claim 20, the combination of Noda et al. in view of Aarnio, as applied to claim 
19, further teaches that the text that is found describing a matching image is communicated to the 
handheld device (page 25, 2.1, end of second paragraph). 

28. As for Claim 21, the combination of Noda et al. in view of Aarnio, as applied to claim 
19, further teaches that the text is returned to the computer that began the search (Figure 4, in 
Aarnio, shows that the information that is found is returned to the mobile system, which is the 
computer that began the search). 

29. As for Claim 22, the combination of Noda et al. in view of Aarnio, as applied to claim 21 
above further teaches the system to provide location awareness services as recited in Claim 21 
wherein the computer that initiated the search is capable of comparing the original image with 
images returned in the computer file having text matching the associated text describing the 
object in the image (see figures 5 and 6, the PDA provides the user with the comparison of 
images that resulted from the original computer, database, to the image). 

30. As for Claim 24, the combination of Noda et al. in view of Aarnio, as applied to claim 
19, further teaches the system to provide location awareness services as recited in Claim 19 
wherein at least one of the computers includes at least one computer file having an image of an 
object of known interest and an associated image of an object of less recognized interest within 
the field of view of the known interest object (section 3.1 describes the extraction of the flower, 
which is the isolation of the image and at the same time the isolation of the surroundings, 
therefore it is inherent that the surroundings can be used to identify an object of less interest 
using the same methods describe in the experimentation with real images 4.1) to aid a user in 
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finding the object of less recognized interest all located within a predetermined radius about a 
geographical location. 

31. As for Claim 27, Noda et al. explicitly teach: The method for identifying a location as 
recited in Claim 26 wherein the comparing step includes comparing at least one of energy 
spectrum data, color histogram data, primitive filter data, and local invariant data (pages 26-38, 
3-3.4 mentions color characteristic extraction). 

32. As for Claim 28, Noda et al. explicitly teach: The method for identifying a location as 
recited in claim 26 wherein the comparing step comprises at least one of the techniques including 
a least square matching technique, a normalizing the image technique, an eigen value technique, 
a matching histogram of image feature technique and an image matching engine with 
transformation technique (pages 27-28, 3.4). 

33. Claim 23 is rejected under 35 U.S.C. 103(a) as being unpatentable over Noda et al. in 
view of Aarnio (US 6,522,889) and further in view of Reed (US 2005/01 131 13 Al). 

34. As for Claim 23, the combination of Noda et al. in view of Aarnio teaches the system to 
provide location awareness services as recited in Claim 15. 

However, the combination does not explicitly teach that the system includes a GPS 
system receiver to identify the geographical location of the mobile communication device and 
help eliminate non-useful images. 

On the other hand, Reed teaches an enhance wireless phone (Figure 2). Reed further 
teaches that this wireless phone has a GPS system 26, which is described in paragraph [0114]. 
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The GPS system locates the user and inherently would provide coordinates that could narrow 
down the image matching process. 

Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include the GPS capabilities of the mobile phone found in Reed into the 
system found in the combination of Noda et al. in view of Aarnio in order to pinpoint exact 
locations and provide the user with the ease of mind of always knowing their location. 



Cited Prior Art 

35. The prior art made of record and not relied upon is considered pertinent to Applicant's 
disclosure for the following reasons: 

o US 7,043,474 B2 discloses a computer-implemented method for characterizing, 
annotating and determining image similarity based on semantic meaning of images. 

o US 6,504,571 Bl discloses a system and method for querying digital image archives 
containing digital photographs and/or videos, where the digital images are indexed in accordance 
with image data such as lens focal length, auto focus distance, shutter speed, exposure duration, 
aperture setting, frame number, image quality, flash status and light meter readings, which are 
used for searching a database consisting of the digital images. 

o US 6,463,426 Bl discloses an image processing system which can rapidly match a 
primary image to a target image - each feature module defines particular regions of an image and 
particular measurements to make on pixels within the defined image region as well as the 
measurements to make on neighboring pixels in neighboring image regions for a given 
application. 
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o US 7,099,860 Bl discloses an image retrieval system performs both keyword-based and 
content-based image retrieval. 

o US 5,852,823 discloses an apparatus and method for generating a semantically based, 
linguistically searchable, numeric descriptor of a pre-defined group of input images. 



Conclusion 

36. Any inquiry concerning this communication or earlier communications from the 
Examiner should be directed to Justin P. Misleh whose telephone number is (571) 272-7313. 
The Examiner can normally be reached Monday - Friday between 8:30 AM - 3:30 PM. 

If attempts to reach the Examiner by telephone are unsuccessful, the Examiner's 
supervisor, David L. Ometz can be reached on (571) 272-7593. The fax phone number for the 
organization where this application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you 
would like assistance from a USPTO Customer Service Representative or access to the 
automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Justin P. Misleh/ 

Primary Examiner, Group Art Unit 2622 
February 2, 2010 



